This Page Is Inserted by IFW Operations 
and is not a part of the Official Record 



BEST AVAILABLE IMAGES 

Defective images within this document are accurate representations of 
the original documents submitted by the applicant. 

Defects in the images may include (but are not limited to): 

• BLACK BORDERS 

• TEXT CUT OFF AT TOP, BOTTOM OR SIDES 

• FADED TEXT 

• ILLEGIBLE TEXT 

• SKEWED/SLANTED IMAGES 

• COLORED PHOTOS 

• BLACK OR VERY BLACK AND WHITE DARK PHOTOS 

• GRAY SCALE DOCUMENTS 

IMAGES ARE BEST AVAILABLE COPY. 



As rescanning documents will not correct images, 
please do not report the images to the 
Image Problem Mailbox. 



(19) 



Europaisches Patentamt 
European Patent Office 
Office europeen des brevets 




(12) 



(11) EP 0 825 782 A2 

EUROPEAN PATENT APPLICATION 



(43) Date of publication: 

25.02.1998 Bulletin 1998/09 



(51) mtci H04N 7/50 



(21) Application number: 97306113.8 

(22) Date of filing: 12.08.1997 



(84) Designated Contracting States: 

AT BE CH DE DK ES R FR GB GR IE IT LI LU MC 
NL PT SE 

(30) Priority: 14.08.1996 JP 214707/96 
30.08.1996 JP 229651/96 

(71) Applicant: SONY CORPORATION 
Tokyo 141 (JP) 

(72) Inventors: 

• Mihara, Kanji 
Shinagawa-ku, Tokyo (JP) 



* Kitamura, Takuya 
Shinagawa-ku, Tokyo (JP) 

• Suzuki, Takao 
Shinagawa-ku, Tokyo (JP) 

(74) Representative: 

Robinson, Nigel Alexander Julian et al 
D. Young & Co., 
21 New Fetter Lane 
London EC4A 1 DA (GB) 



(54) Video data compression apparatus and method 



(57) A video data compression apparatus (1) with 
which compressed video data (VOUT) of suitable 
amounts of data can be produced and in addition the 
time required for processing is short. 

An encoder control unit (1 2) performs a preprocess- 
ing for compression and coding and, at the same time, 
produces a flatness and an intra AC as parameters in- 
dicating the difficulty of the pattern of pictures to be com- 
pressed to I pictures. A motion detector (14) calculates 
a predictive error amount (ME residual) of the motion 



prediction. An FIFO memory (160) delays each picture 
of the input video data. 

A host computer (20) approximates a real difficulty 
data Dj indicating the difficulty of the pattern of each pic- 
ture by the ME residual, flatness, and intra AC and fur- 
ther calculates the target amount of data Tj of the com- 
pressed video data from the approximated real difficulty 
data Dj. The encoder (18) performs the compression 
and coding so that the amount of data of the com- 
pressed video data becomes substantially the target 
amount of data T,. 
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Description 

The present invention relates to a video data compression apparatus for compressing and coding noncompressed 
video data and a method of same. 

When compressing and coding noncompressed digital video data in units of GOPs (groups of pictures) comprised 
oi an I picture (intra-coded picture), a B picture (bi-directionally predictive coded picture), and a P picture (predictive 
coded picture) by a method such as the MPEG (Moving Picture Experts Group) and recording the same on a recording 
mcdum such as a magneto-optical disc (MO disc) or transmitting the same via communication line, it is necessary to 
control the amount of data (amount of bits) of the compressed video data after the compression and coding to be not 
m> c than the recording capacity of the recording medium or not more than the transmission capacity of the commu- 
nicrtion line while holding the quality of the image after expansion and decoding high. 

For this purpose, first, there is adopted a method of preliminarily compressing and coding the noncompressed 
vooo OntH lo estimate the amount of data after the compression and coding (first pass), then adjusting the compression 
frdo based on the estimated amount of data and performing the compression and coding so that the amount of data 
after ir>c compression and coding becomes not more than the recording capacity of the recording medium by (second 
p<is4t (rierctrvittc such a compression and coding method will be also referred to as "two pass encoding'). 

Mowcvcr it performing the compression and coding by the two pass encoding, it is necessary to apply similar 
compression and coding processing two times with respect to the same noncompressed video data, so a long time :3 
taken Further since the final compressed video data cannot be calculated by one compression and coding p recess =;ig, 
trie snot video data cannot be compressed, coded, recorded, or transmitted in real time as it is. 

An object of the present invention is to provide a video data compression apparatus with which the audio and/or 
video data can be compressed and coded to a predetermined amount of data or less not by the two pass encoding 
and a method ot tho same. 

Further another object of the present invention is to provide a video data compression apparatus with which the 
video data can be compressed and coded in almost real time and in addition with which a high quality image can be 
obtained after the expansion and decoding and a method of the same. 

Further, still another object of the present invention is to provide a video data compression apparatus with which 
the compression and coding processing can be carried out by adjusting the compression rate by estimating the amount 
of data after the compression and coding not by the two pass encoding and a method of the same. 

According to a first aspect of the present invention, there is provided a video data compression apparatus having 
an indicator data calculating means for calculating indicator data indicating a complexity of video data for every picture 
from noncompressed video data; a target value calculating means for calculating a target value of an amount of data 
after compression of the video data for every picture based on the calculated indicator data; and a compressing means 
for compressing the noncompressed video data so that the amount of data after compression becomes the calculated 
target value. 

According to a second aspect of the present invention, there is provided a data compression method comprising 
the steps of calculating indicator data indicating a complexity of video data for every picture from noncompressed video 
data; calculating a target value of an amount of data after compression of the video data for every picture based on 
the calculated indicator data; and compressing the video data by a predetermined compression method so that the 
amount of data after compression becomes the calculated target value. 

According to a third aspect of the present invention, there is provide a video data compression apparatus having 
an indicator data calculating means for calculating indicator data indicating a complexity of video data for every picture; 
a difficulty data calculating means for performing a predetermined computation processing for multiplying a coefficient 
with the calculated indicator data to calculate difficulty data corresponding to the amount of data after compression of 
the video data; a target value calculating means for calculating a target value of the amount of data after compression 
of the video data for every picture based on the calculated difficulty data; a compressing means for compressing each 
of the pictures of the video data by a predetermined compression method so that the amount of data after compression 
becomes the calculated target value so as to generate compressed video data; and a coefficient updating means for 
updating the coefficient based on the amount of data of the generated compressed video data. 

According to a fourth aspect of the present invention, there is provided a video data compression method com- 
prising the steps of calculating indicator data indicating a complexity of video data for every picture; performing pre- 
determined computation processing for multiplying a coefficient with the calculated indicator data to calculate difficulty 
data corresponding to the amount of data after compression; calculating a target value of the amount of data after 
compression of the noncompressed video data for every picture based on the calculated difficulty data; compressing 
each of the pictures of the video data by the compression method so that the amount of data after compression becomes 
the calculated target value so as to generate compressed video data; and updating the coefficient based on the amount 
of data of the generated compressed video data. 

According to a fifth aspect of the present invention, there is provided a video data compression apparatus for 
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compressing a continuous plurality of video data to compressed video data of a picture type sequence containing a 
plurality of types of pictures (I picture, P picture, and B picture) in a predetermined order, having a rearranging means 
for rearranging pictures of the noncompressed video data to an order adapted to the compression method so that each 
head picture of the video data becomes an I picture or a P picture: an indicator data calculating means for calculating 
5 indicator data indicating a complexity o1 the rearranged noncompressed video data for every picture: a border detecting 
means for detecting a scene change of a continuous plurality of the noncompressed video data; a changing means 
for changing the picture type sequence so that pictures of any of the noncompressed video data are compressed 
without reference to the pictures of the other noncompressed video data for every border of a detected scene change; 
a target value calculating means for calculating a target value of the amount of data after compression of the video 
io data based on the calculated indicator data and the picture type sequence after change; and a compressing means 
for compressing the video data to compressed video data of the picture type sequence after change so that the amount 
of data after compression becomes substantially the calculated target value. 

According to a sixth aspect of the present invention, there is provided a video data compression method tor com- 
pressing a continuous plurality of video data to compressed video data of a picture type sequence containing a plurality 
is of types of pictures (I picture, P picture, and B picture) in a predetermined order, comprising the steps of rearranging 
pictures of the noncompressed video data to an order adapted to the compression method so that each head picture 
of the video data becomes an I pictu re or a P picture; calculating indicator data indicating a complexity of the rearranged 
noncompressed video data for every picture; detecting-a scene change of the continuous plurality of the noncom- 
pressed video data; changing the picture type sequence so that a picture of any of the noncompressed video data is 
20 compressed without reference to a pictures of other noncompressed video data for every border of a detected scene 
change; calculating a target value of the amount of data after compression of the video data for every predetermined 
compression unit based on the calculated indicator data and the picture type sequence after change; and compressing 
the video data to the compressed video data of the picture type sequence after change so that the amount of data after 
compression becomes substantially the calculated target value. 
25 Note that flatness is defined as an indicator representing a spatial flatness of van image, represents the complexity 

of the image, and has a correlation with the difficulty (degree of difficulty) of the pattern of the image and the amount 
of data after compression. 

Further, an intra AC is defined as the sum of the dispersion values of the average value of the video data in the 
pictures and the video data for every DCT block of the DCT processing unit in the MPEG system, represents the 
30 complexity of the image, and has a correlation with the difficulty (degree of difficulty) of the pattern of the image and 
the amount of data after compression similar to the flatness. 

Further, an ME residual is defined as the sum of absolute values or the sum of square values of the difference of 
the video data between the input picture and a reference picture after motion compensation processing by-the motion 
vector at the compression and coding, represents the speed of the motion of the image and the complexity of the 
35 pattern, and has a correlation with the difficulty (degree of difficulty) of the pattern of the image and the amount of data 
after compression similar to the flatness. 

Further, the activity is defined as the activity used for determining a quantization value of quantization processing 
with respect to a macroblock in TM5 [test model 5; I SO/I EC JTC/SC29 (1 993)] famous as a compression algorithm of 
the MPEG system. 

40 Embodiments of the invention will now be described, by way of example only, with reference to the accompanying 

drawings in which: 

Fig. 1 is a view of the configuration of a video data compression apparatus according to the present invention; 
Fig. 2 is a view of the configuration of an encoder of a simplified two pass processing unit shown in Fig. 1 ; 
45 Fig. 3 is a view of the configuration of the encoder shown in Fig. 1 ; 

Figs. 4A to 4C are views of an operation of the simplified two pass encoding of the video data compression appa- 
ratus according to a first embodiment of the present invention 

Fig. 5 is a schematic view of the configuration of the video data compression apparatus according to a second 
embodiment of the present invention; 
so Fig. 6 is a detailed view of the configuration of a compression and coding unit of the video data compression 

apparatus shown in Fig. 5; 

Fig. 7 is a view of the correlation between an ME residual and a real difficulty data Dj when producing a P picture 
by the video data compression apparatus shown in Fig. 1 and Fig. 5; 

Fig. 8 is a view of the correlation between the ME residual and the real difficulty data Dj when producing a B picture 
55 by the video data compression apparatus shown in Fig. 1 and Fig. 5; 

Fig. 9 is a view of a method of calculation of flatness; 

Fig. 10 is a view of the correlation between the flatness and the real difficulty data Dj when producing an I picture 
by the video data compression apparatus shown in Fig. 1 and Fig. 5; 
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Fig. 11 is a view of the correlation between the flatness and the real difficulty data Dj when producing an I picture 
by the video data compression apparatus shown in Fig. 1 and Fig. 5; 

Fig. 1 2 is a view of a compression and coding operation of the video data compression apparatus (Fig. 6) according 
to a third embodiment of the present invention; 
s Fig. 1 3 is a view of the content of the processing of the host computer (Fig. 6) of the video data compression 

apparatus 2 according to the third embodiment of the present invention; 

Fig. 14 is a view of the compression and coding operation of the video data compression apparatus according to 
a fourth embodiment of the present invention in an order of coding; 

Figs. 1 5 A to 1 5C are views of a processing for rearrangement of the order of the pictures of the edited video data 
w by an encoder control unit (Fig. 1 ) and processing for changing the picture type by the host computer; and 

Fig. 1 6 is a flowchart of the content of the processing of the host computer of the video data compression apparatus 
according to a -fourth embodiment of the present invention 

First Embodiment 

75 

Below, a first embodiment of the present invention will be explained. 

If a plurality of video data having a high coding difficulty such as a pattern having a large number of high frequency 
components or a pattern having a large amount of motion are compressed and coded by a compression and coding 
system of video data such as the MPEG system, generally distortion accompanying the compression is apt to occur. 
20 For this reason, it is necessary to compress and code video data having a high difficulty with a low compression rate 
and allocate a larger target amount of data with respect to the compressed video data obtained by compressing and 
coding the data having a high difficulty in comparison with compressed video data of video data of a pattern having a 
low difficulty. 

In order to adaptively allocate the target amount of data with respect to the difficulty of the video data in this way, 
2S the two pass encoding system indicated as the related art is effective. However, the two pass encoding system is not 
suited to real time compression and coding. 

The simplified two pass encoding system indicated as the first embodiment was devised so as to solve the disad- 
vantage of such a two pass encoding system. It can calculate the difficulty of the noncompressed video data from the 
difficulty data of the compressed video data obtained by preliminarily compressing and coding the noncompressed 
30 video data and adaptively control the compression rate of the noncompressed video data delayed by exactly a prede- 
termined time by an FIFO memory or the like based on the difficulty calculated by the preliminary compression and 
coding. 

Figure 1 is a view of the configuration of a video data compression apparatus 1 according to the present invention. 
As shown in Fig. 1 t the video data compression apparatus 1 is constituted by a compression and coding unit 10 
35 and a host computer 20. The compression and coding unit 10 is constituted by an encoder control unit 12, a motion 
detector 14, a simplified two pass processing unit 16, and a second encoder 18. The simplified two pass processing 
unit 16 is constituted by an FIFO memory 160 and a first encoder 162. 

The video data compression apparatus 1 realizes the above simplified two pass encoding with respect to the 
noncompressed video data VIN input from an external apparatus (not illustrated) such as an editing apparatus and a 
40 video tape recorder apparatus by these constituent parts. 

In the video data compression apparatus 1, the host computer 20 controls the operation of the constituent parts 
of the video data compression apparatus 1 . Further, the host computer 20 receives the amount of data of the com- 
pressed video data generated by preliminarily compressing and coding the noncompressed video data VIN by the 
encoder 162 of the simplified two pass processing unit 16, the value of the direct current component (DC component) 
45 of the video data after the DCT processing, and the power value of the alternating current component (AC component) 
via a control signal C1 6 and calculates the difficulty of the pattern of the compressed video data based on these received 
values. Further, the host computer 20 allocates the target amount of data Tj of the compressed video data generated 
by the encoder 18 via the control signal C18 for every picture based on the calculated difficulty, sets the same in the 
quantization circuit 166 (Fig. 3) of the encoder 18, and adaptively controls the compression rate of the encoder 18 in 
so units of pictures. 

The encoder control unit 12 notifies the existence of a picture of the noncompressed video data VIN to the host 
computer 20 and further performs the preprocessing for the compression and coding for every picture of the noncom- 
pressed video data VIN. That is, the encoder control unit 12 rearranges the input noncompressed video data in the 
order of coding, performs picture field transformation, performs 3:2 pull down processing (processing for transforming 
55 the video data of 30 frames/sec to video data of 24 frames/sec), etc. when the noncompressed video data VIN is the 
video data of a movie and outputs the same as the video data S12 to the FIFO memory 160 and the encoder 162 of 
the simplified two pass processing unit 16. 

The motion detector 14 performs the detection of the motion vector of the noncompressed video data and outputs 
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the same to the encoder control unit 12 and the encoders 162 and 18. 

In the simplified two pass processing unit 16. the FIFO memory 160 delays the video data S12 input from the 
encoder control unit 12 by exactly a time by which for example L (L is an integer) number of pictures of the noncom- 
pressed video data VIN is input and outputs the same as the delayed video data S1 6 to the encoder 1 8. 
5 Figure 2 is a view of the configuration of the encoder 162 of the simplified two pass processing unit 16 shown in 

Fig- 1 - 

The encoder 162 is a video data use compression and coding unit constituted by, for example, as shown in Fig. 
2, an adder circuit 164, a DCT circuit 166, a quantization circuit (Q) 168, a variable length coding circuit (VLC) 170, an 
inverse quantization circuit (IQ) 172, an inverse DCT (IDCT) circuit 174, an adder circuit 176, and a motion compen- 
io sation circuit 178, compresses and codes the input video data S12 by the MPEG system or the like, and outputs the 
amount of data etc. for every picture of the compressed video data to the host computer 20. 

The adder circuit 1 64 subtracts the output data of the adder circuit 1 76 from the video data S 1 2 and outputs the 
result to the DCT circuit 166. 

The DCT circuit 166 performs discrete cosine transformation (DCT) processing on the video data input from the 
is adder circuit 164 in units of DCT blocks consisting of for example 8 pixels x 8 pixels, transforms the same from data 
of the time domain to data of the frequency domain, and outputs the result to the quantization circuit 168. Further, the 
DCT circuit 166 outputs the value of the DC component of the video data after DCT and the power value o1 the AC 
component to the host computer 20. 

The quantization circuit 1 68 quantizes the data of the frequency domain input from the DCT circuit 1 66 by the fixed 
20 quantization value Q and outputs the result as the quantized data to the variable length coding circuit 170 and the 
inverse quantization circuit 172. 

The variable length coding circuit 170 performs variable length coding on the quantized data input from the quan- 
tization circuit 168 and outputs the amount of data of the compressed video data obtained as a result of the variable 
length coding via the control signal C16 to the host computer 20. 
25 The inverse quantization circuit 172 performs inverse quantization on the quantized data input from the variable 

length coding circuit 168 and outputs the same as the inverse quantized data to the inverse DCT circuit 174. 

The inverse DCT circuit 1 74 performs inverse DCT processing on the inverse quantized data input from the inverse 
quantization circuit 172 and outputs the result to the adder circuit 176. 

The adder circuit 176 adds the output data of the motion compensation circuit 178 and the output data of the 
30 inverse DCT circuit 174 and outputs the result to the adder circuit 164 and the motion compensation circuit 178. 

The motion compensation circuit 178 performs the motion compensation processing based on the motion vector 
input from the motion detector 1 4 on the output data of the adder circuit 176 and outputs the result to the adder circuit 
176. 

Figure 3 is a view of the configuration of the encoder 18 shown in Fig. 1 . 

35 As shown in Fig. 3, the encoder 18 is configured as the encoder 162 shown in Fig. 2 plus the quantization control 

circuit 180. The encoder 18 applies the motion compensation processing, DCT processing, quantization processing, 
and variable length coding processing with respect to the delayed video data SI 6 delayed by the amount of L pictures 
by the FIFO memory 160 by these constituent parts based on the target amount of data Tj set from the host computer 
20 to generate the compressed video data VOUT of the MPEG system or the like and outputs the result to an external 

40 apparatus (not illustrated). 

In the encoder 1 B, the quantization control circuit 180 successively monitors the amount of data of the compressed 
video data VOUT output by the variable length quantization circuit 170 and successively adjusts the quantization value 
q. to be set in the quantization circuit 168 so that the amount of data of the compressed video data finally generated 
from the j-th picture of the delayed video data S1 6 approaches the target amount of data Tj set from the host computer 

45 20. 

Further, the variable length quantization circuit 170 outputs the compressed video data VOUT to the outside and 
in addition outputs the actual amount of data Sj of the compressed video data VOUT obtained by compressing and 
coding the delayed video data S1 6 via the control signal C18 to the host computer 20. 

Below, an explanation will be made of the simplified two pass encoding operation of the video data compression 
50 apparatus 1 in the first embodiment. 

Figures 4A to 4C are views of the operation of the simplified two pass encoding of the video data compression 
apparatus 1 in the first embodiment 

The encoder control unit 1 2 performs the preprocessing of rearrangement of pictures in the order of coding etc. 
with respect to the noncompressed video data VIN input to the video data compression apparatus 1 by the encoder 
55 control unit 1 2 and outputs the same as the video data S12 to the FIFO memory 160 and the encoder 162 as shown 
in Fig. 4A. 

Note that, due to the rearrangement of the order of pictures by the encoder control unit 12. the order of coding of 
the pictures shown in Figs. 4A to 4C etc. and the order indicated after expansion and decoding are different. 
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The FIFO memory 160 delays the pictures of the input video data SI 2 by exactly the amount of L pictures and 
outputs the same to the encoder 18. 

The encoder 162 preliminarily successively compresses and codes the pictures of the input video data S12 and 
outputs the amount of data of the compressed and coded data obtained by compressing and coding the j 0 is an 

5 integer)-th picture, the value of the DC component of the video data after the DCT processing, and the power value of 
the AC component to the host computer 20. 

For example, since the delayed video data S16 input to the encoder 18 is delayed by exactly the amount of L 
pictures by the FIFO memory 160, as shown in Fig. 4B, when the encoder 18 compresses and codes the j (j is an 
integer)-th picture (picture a of Fig. 4B) of the delayed video data S16, the encoder 162 compresses and codes the 

10 (j+L)-th picture (picture b of Fig. 4B) advanced from the j-th picture of the video data S1 2 by the amount of L pictures. 
Accordingly, when the encoder 1 8 starts the compression and coding of the j-th picture of the delayed video data S16, 
the encoder 1 62 has completed the compression and coding of the j-th to (j+L-1 )-th pictures (range c of Fig. 4B) of the 
video data S1 2, and the real difficulty data Dj, D K1 , D^ 2 . — . D^.-, after compression and coding of these pictures have 
been already calculated by the host computer 20. 

is The host computer 20 calculates the target amount of data Tj to be allocated to the compressed video data obtained 

by compressing and coding the j-th picture of the delayed video data S1 6 by the encoder 1 8 by the following Equation 
1 and sets the calculated target amount of data Tj in the quantization control circuit 180. 

20 t=r — — — (I) 

j Jj-i.-i 

£ £\ 

2S Note, in Equation 1 , Dj is the real difficulty data of the j-th picture of the video data S12, R'j is an average of the 

target amount of data which can be allocated to the j-th to the (j+L-1 )-th pictures of the video data S12 and S16, and 
an initial value (R\ ) of R'j is the target amount of data which can be allocated to pictures of the compressed video data 
on the average, represented by following Equation 2, and updated as shown in Equation 3 whenever the encoder 18 
generates one picture's worth of the compressed video data. 

30 

H\ = (Bit_rate x L)/Picture_rate (2) 

R Vi- R r s i + F i + L < 3 > 

Note that, the numerical value bit rate (Bit rate) in Equation 3 indicates the amount of data (amount of bits) per 
second determined based on the transmission capacity of the communication line and the recording capacity of the 
recording medium, "Picture rate" indicates the number of pictures per second contained in the video data (30 pictures/ 
40 sec (NTSC), 25 pictures/sec (PAL)), and the numerical value F j+L indicates the average amount ot data per picture 
determined in accordance with the picture type. 

The DCT circuit 1 66 of the encoder 1 8 performs DCT processing on the j-th picture of the input delayed video data 
S16 and outputs the result to the quantization circuit 168. 

The quantization circuit 168 quantizes the data of the frequency domain of the j-th picture input from the DCT 
circuit 166 by the quantization value Qj adjusted based on the target amount of data Tj by the quantization control 
circuit 180 and outputs the result as the quantized data to the variable length coding circuit 170. 

The variable length coding circuit 170 performs variable length coding on the quantized data of the j-th picture 
input from the quantization circuit 168 so as to generate the compressed video data VOUT of an amount of data near 
the target amount of data Tj and outputs the same. 

Similarly, as shown in Fig. 4B, when the encoder 18 compresses and codes the (j+1)-th picture (picture a* of Fig. 
4C) of the delayed video data Si 6, the encoder 162 completes the compression and coding of the (j+l)-th to (j+L)-th 
pictures (range c* of Fig. 4C) of the video data S12. The real difficulty data D j+1 , D^ 2 , D j+3 , D| +L-1 of these pictures 
have been already calculated by the host computer 20. 

The host computer 20 calculates the target amount of data T j+1 to be allocated to the compressed video data 
55 obtained by compressing and coding the (j+1 )-th picture of the delayed video data S16 by the encoder 18 by Equation 
1 and sets the same in the quantization control circuit 180 of the encoder 18. 

The encoder 18 compresses and codes the (j+1 )-th picture based on the target amount of data Tj set in the quan- 
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tization control circuit 1 SO from the host computer 20 so as to generate the compressed video data VOUT of an amount 

of data near the target amount of data T^ and outputs the same. 

Further, below, similarly, the video data compression apparatus 1 successively compresses and codes the k-th 

picture of the delayed video data S16 by changing the quantization value 0* (k = j+2, j+3. ™) for every picture and 
5 outputs the result as the compressed video data VOUT. 

As explained above, according to the video data compression apparatus 1 shown in the first embodiment, the 

difficulty of the pattern of the noncompressed video data VIN can be calculated in a short time and the noncompressed 

video data VIN can be adapfrvefy compressed and coded with a compression rate in accordance with the calculated 

difficulty. That is, according to the video data compression apparatus 1 shown in the first embodiment, the noncom- 
10 pressed video data VIN can be adaptively compressed and coded in almost real time based on the difficulty of the 

pattern of the noncompressed video data VIN unlike the two pass encoding system, and thus the apparatus can be 

applied to applications such as live broadcasts in which a real time property is required. 

Note that, in addition to that shown in the first embodiment, the data video data compression apparatus 1 according 

to the present invention can adopt various other configurations, for example, the amount of data of the compressed 
is video data compressed and coded by the encoder 1 62 may be used as it is as the difficulty data so as to simplify the 

processing of the host computer 20. 

Second Embodiment 

20 Below, a second embodiment of the present invention will be explained. 

The simplified two pass encoding system shown in the first embodiment is an excellent system capable of per- 
forming compression and coding by giving only a delay of about one GOP (for example 0.5 second) to the input non- 
compressed video data and generating compressed video data of suitable amounts of data. 

In this system, however, two encoders are required. In general, an encoder for compressing and coding video data 
25 requires large hardware and is very expensive even if formed as an integrated circuit. In addition, it is large in size. 
Accordingly, the necessity of two encoders in this system prevents a reduction of the cost of the apparatus, a reduction 
of size, and saving of electric power. Further, it is desirable that the time delay required for the compression and coding 
be as short as possible s but the processing for calculation of the real difficulty data Dj and the predictive difficulty data 
D/ and the preliminary compression and coding processing per se require a few pictures 1 worth of processing time, 
30 therefore these processings per se become a factor preventing the shortening of the time delay 

The second embodiment was made so as to solve such a disadvantage and provides a video data compression 
method capable of generating compressed video data of suitable amounts of data equivalent to the simplified two pass 
encoding system by only using one encoder and in addition requiring a shorter time delay for the processing. 

Figure 5 is a schematic view of the configuration of a video data compression apparatus 2 according to a second 
35 embodiment of the present invention. 

Figure 6 is a detailed view of the configuration of a compression and coding unit 24 of the video data compression 
apparatus 2 shown in Fig. 5. 

Note that, in Fig. 5 and Fig. 6, among the constituent parts of the video data compression apparatus 2, the parts 
the same as the constituent parts of the video data compression apparatus 1 explained in the first embodiment (Fig. 
40 1 and Fig. 2) are indicated by same symbols or numerals. 

As shown in Fig. 5, the video data compression apparatus 2 adopts a configuration in which the compression and 
coding unit 10 of the video data compression apparatus 1 (Fig. 1 and Fig. 2) is replaced by a compression and coding 
unit 24 obtained by excluding the encoder 1 62 from the compression and coding unit 1 0, replacing the encoder control 
unit 12 by the encoder control unit 22, and adding a buffer memory (buffer) 182. 
45 As shown in Fig. 6, the compression and coding unit 24 is constituted by an image rearranging circuit 220, a 

scanning transforming and macroblock forming circuit 222, and a statistical amount (parameter) calculation circuit 224. 
The rest of the constituent parts of the compression and coding unit 24 adopt the same configurations as those of the 
compression and coding unit 10. 

The encoder control unit 22 notifies the existence of a picture of the noncompressed video data VIN to the host 
so computer 20 similar to the encoder control unit 1 2 and further performs the preprocessing for compression and coding 
for every picture of the noncompressed video data VIN. 

In the encoder control unit 22, the image rearranging circuit 220 rearranges the input noncompressed video data 
in the order of coding. 

The scanning transforming and macroblock forming circuit 222 performs the picture field transformation and per- 
55 forms the 3:2 pull down processing etc. where the source of the noncompressed video data VIN is the video data of a 
movie. 

The statistical amount calculation circuit 224 calculates a statistical amount such as the flatness and intra AC from 
the pictures processed by the image rearranging circuit 220 and the scanning transforming and macroblock forming 
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circuit 222 and compressed and coded to I pictures. 

The video data compression apparatus 2 uses a statistical amount (flatness, intra AC) of the noncompressed video 
data and the predictive error amount (ME residual) of motion prediction in place of the difficulty of the pattern of the 
noncompressed video data VIN by these constituent parts, adaptively calculates the target amount of data Tj similar 
5 to the video data compression apparatus 1 (Fig. 1 and Fig. 2), and performs high precision feed forward control to 
compress and code the noncompressed video data VIN to compressed video data of suitable amounts of data. 

Note that, in the video data compression apparatus 2, the target amount of data T| is determined by the motion 
detector 1 4 and the statistical amount calculation circuit 224 of the encoder control unit 22 based on the indicator data 
(parameter) detected in advance. Therefore, below, the compression and coding system in the video data compression 
10 apparatus 2 will be referred to as a feed forward rate control (FFRC) system. 

Note that, the ME residual is defined as the sum of the absolute values or the sum of the square values of the 
difference of the video data between pictures to be compressed and a reference picture, is calculated from the pictures 
which become the P pictures and B pictures after compression by the motion detector 1 4, represents the speed of the 
motion of the image and the complexity of the pattern, and has a correlation with the difficulty and the amount of data 
is after compression similar to the flatness. 

The I picture is compressed and coded without reference to other pictures, so the ME residual cannot be found. 
Therefore, the flatness and intra AC are used as the parameters in place of the ME residual. 

Further, flatness is a parameter newly defined as an indicator representing the spatial flatness of the image so as 
to realize the video data compression apparatus 2, indicates the complexity of the video, and has a correlation with 
20 the difficulty (degree of difficulty) of the pattern of the image and the amount of data after the compression. 

Further, the intra AC is a parameter newly defined as the sum of the dispersion values of the video data for every 
OCT block of the DCT processing unit in the MPEG system so as to realize the video data compression apparatus 2, 
indicates the complexity of the image, and has a correlation with the difficulty of the pattern of the image and the amount 
of data after the compression similar to the flatness. 
25 Below, an explanation will be made of the ME residual, flatness, and intra AC. 

In the simplified two pass encoding system explained in the first embodiment, the real difficulty data Dj indicated 
the difficulty of the pattern of the image and the target amount of data Tj was calculated based on the real difficulty 
data Dj . 

In order to make the amount of data of the compressed video data generated by the encoder 18 approach the 
30 value indicated by the target amount of data Tj, the quantization value Qj is controlled in the quantization circuit 168 
(Fig. 2 and Fig. 6). 

Accordingly, if a parameter obtained without compressing and coding the video data and adequately indicating the 
complexity (difficulty) of the video data similar to the real difficulty data Dj can be obtained before the quantization 
processing in the quantization circuit 1 68 of the encoder 1 8, it would be possible to achieve the objectives of eliminating 
35 the encoder 162 (Fig. 1) and shortening the processing delay time. The ME residual, the flatness, and the intra AC 
have a strong correlation with the real difficulty data Dj and therefore are adequate for achieving these objectives. 

Relationship Between ME Residual and Real Difficulty Data D j 

40 When performing compression and coding processing by referring to other pictures to generate the P pictures and 

B pictures, the motion detector 14 searches for a macroblock giving the smallest sum of the absolute values or sum 
of the square values of the difference between a particular macroblock of the picture (input picture) to be compressed 
and the picture referred to (reference picture) so as to find the motion vector. The ME residual is defined as the value 
obtained by adding up the sums of absolute values or sums of the squares of the differences of each of macroblocks 

45 giving the smallest sums for all of the pictures when finding the motion vector in this way. 

Figure 7 is a view of the correlation between the ME residual and the real difficulty data Dj when generating a P 
picture by the video data compression apparatuses 1 and 2. 

Figure 8 is a view of the correlation between the ME residual and the real difficulty data Dj when generating a B 
picture by the video data compression apparatuses 1 and 2. 

so Note that, in Fig. 7 and Fig. 8, as the real difficulty data Dj, use is made of the amount of data of the compressed 

video data obtained by compression and coding by using a fixed quantization value by the encoder 18 (below, the 
same in Fig. 10 and Fig. 11 ), and Fig. 7 and Fig. 8 are graphs of the relationship between the ME residual and the real 
difficulty data Dj obtained where standard images ["cheer" (cheer leaders), "mobile" (mobile and calendar), "tennis" 
(table tennis), and "diva 0 (diva with noise)] standardized by the CCIR and other images ("resort") are actually com- 

55 pressed and coded by the MPEG system. In Fig. 7 and Fig. 8, an ordinate of the graph (difficulty) indicates the real 
difficulty data Dj, and abscissa (me resid) indicates the ME residual. 

As understood when referring to Fig. 7 and Fig. 8 : the ME residual has a strong correlation with the real difficulty 
data Dj. Accordingly, in place of the real difficulty data Dj of a picture which becomes a P picture or a B picture after 
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compression, the ME residual can be used for the generation of the target amount of data Tj . 

Relationship between Flatness and Real Difficulty data D j 

5 Figure 9 is a view of the method of calculation of the flatness. 

The flatness is calculated by dividing each of the DCT blocks which serve as the units of DCT processing in the 
MPEG system into small blocks each consisting of 2 pixels x 2 pixels as shown in Fig. 9, calculating the difference of 
the data of the diagonal pixels (pixel values) in these small blocks, comparing the difference with a predetermined 
threshold value, and further finding the sum of small blocks in which the difference becomes smaller than the threshold 
io value for every picture. 

Note that the value of the flatness becomes smaller as the pattern of the video becomes spatially more complex 
and becomes larger when it is flatter. 

Figure 10 is a view of the correlation between the flatness and the real difficulty data Dj when generating an I 
picture by the video data compression apparatuses 1 and 2. 
is Note that, Fig. 10 is a graph of the relationship between the flatness and the reaJ difficulty data Dj obtained when 

the standard images standardized by the CCIR and other images are actually compressed and coded by the MPEG 
system. In Fig. 10, the ordinate of the graph (difficulty) indicates the real difficulty data D j( and the abscissa (flatness) 
indicates the flatness. 

As shown in Fig. 10, there is a strong negative correlation between the flatness and the real difficulty data D j: and 
20 it is seen that the real difficulty data Dj can be approximated by the method of substituting the flatness into the primary 
function etc. 
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Relationship Between Intra AC and Real Difficulty Data D, 



25 The intra AC is calculated as the sum of the absolute values of the difference between the pixel value of each pixel 

in a DCT block (8x8 pixels) and the average value (£) of the pixel values (Pk) in the DCT block for every DCT block. 
That is, the intra AC can be found by the following equation 4. 



lntra-AC r £ Y,\ p r p \ 

ptcxvr* t-l 

(4) 

- 1 ^ 

64 



Figure 11 is a view of the correlation between the intra AC and real difficulty data Dj when generating an I picture 
by the video data compression apparatuses 1 and 2. 

Note that, Fig. 11 is a graph showing the relationship between the intra AC and the real difficulty data Dj obtained 
40 where the standard images standardized by the CCIR and other images are actually compressed and coded by the 
MPEG system. In Fig. 11, the ordinate of the graph (difficulty) indicates the real difficulty data Dj, and the abscissa 
(intra AC) indicates the intra AC. 

As explained heretofore, it is seen that the real difficulty data Dj can be approximated by the primary function etc. 
by each indicator data (statistical amount). Accordingly, it is possible to calculate the real difficulty data Dj of each 
45 picture type as follows. 

The real difficulty data Dj is approximated by the ME residual (MEj) by the following Equation 5 for a P picture and 
by the following Equation 6 for a B picture. Further, the real difficulty data Dj is approximated by the flatness (FRj) and 
intra AC (Intra ACj) or either of them by approximation equations (Equations 7 and 8) similar to Equations 5 and 6 for 
an I picture. 



Dj^apXMEj + bp (5) 



D i = a B xME j+ b B (6) 
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D^ajXFRj+b, (7) 

D| = a 1 , x (intra ACp + b', (8) 
Here, a P , a B , a,, and a,' are proportional coefficients. 

Further, in the simplified two pass encoding system shown in the first embodiment, the target amount of data Tj 
was calculated by substituting the real difficulty data Dj obtained by these approximations into Equation 1 . 

Below, the operation of the video data compression apparatus 2 will be explained by taking as an example a case 
where the real difficulty data Dj is approximated by the ME residual, the flatness, and the intra AC and the noncom- 
pressed video data is compressed and coded by the simplified two pass encoding system. 

In the encoder control unit 22, the image rearranging circuit 220 rearranges the noncompressed video data VIN 
in pictures in the order of coding. The scanning transformation and macroblock forming circuit 222 performs the picture 
field transformation etc. The statistical amount calculation circuit 224 performs the computation processing shown in 
Fig. 9 and Equation 4 with respect to a picture to be compressed and coded to an I picture and calculates a statistical 
amount such as the flatness and intra AC. 

The motion detector 1 4 generates motion vectors for the pictures to be compressed and coded to a P picture and 
B picture and further calculates the ME residual. 

The FIFO memory 160 delays the input video data by exactly an amount of L pictures. 

The host computer 20 performs the computation processing shown in Equation 7 and Equation 8 with respect to 
the ME residual generated by the motion detector 14 to approximate the real difficulty data Dj and performs the com- 
putation processing similar to Equation 7 and Equation 8 to approximate the real difficulty data Dj by the flatness and 
the intra AC. 

Further, the host computer 20 substitutes the approximated real difficulty data Dj into Equation 1 to calculate the 
target amount of data Tj and sets the calculated target amount of data Tj in the quantization control circuit 180 of the 
encoder 18. 

The DCT circuit 1 66 of the encoder 1 8 performs DCT processing on the j-th picture of the delayed video data. 

The quantization circuit 168 quantizes the data of the frequency domain of the j-th picture input from the DCT 
circuit 166 by the quantization value Qj adjusted by the quantization control circuit 180 based on the target amount of 
data Tj. 

The variable length coding circuit 170 performs variable length coding on the quantized data of the j-th picture 
input from the quantization circuit 168, generates the compressed video data VOUT of an amount of data near the 
target amount of data T jt and outputs the same via the buffer memory 182 to an external unit. 

Note that, in the TM5 system or the like known as the compression algorithm of the MPEG, to calculate the quan- 
tization value of the macroblock (MQUANT), a statistical amount such as the "activity' shown in the following Equation 
9 is used. The activity has a strong correlation with the real difficulty data Dj similar to the flatness and the intra AC, 
therefore it is also possible to constitute the video data compression apparatus 2 so as to perform the compression 
and coding by approximating the rea! difficulty data Dj using the activity in place of these parameters. 

act^X+mxn^-l^Jyar sblk) 

(9) 

var sblk^Y, {P,' 1 ^ 

Where, 

actj: j-th activity 
sblk: sub block 
var: dispersion. 

Further, while the operation of the video data compression apparatus 2 was explained by taking as an example a 
case where the simplified two pass encoding shown in the first embodiment was carried out, needless to say the video 
data compression apparatus 2 can perform the predictive simplified two pass encoding as well. 

Further, modifications similar to those with respect to the video data compression apparatus 1 shown in the first 
embodiment are also possible with respect to the video data compression apparatus 2 shown in the second embodi- 
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ment. 

Third Embodiment 

5 Below a third embodiment of the present invention will be explained. 

in the FFRC system shown in the second embodiment, the real difficulty data Dj was approximated by substituting 
tnc statically found indicator data (statistical amount), that is, the ME residual, flatness, intra AC, and activity into 
tnc pf imary function such as Equation 5 and Equation 6. 

Those indicator data and the real difficulty data D ] have a strong correlation as shown in Fig. 7, Fig. 8, Fig. 10. and 
»c f n but a slight difference occurs-f rom the above primary function according to the pattern of the video data. 

The processing of the video data compression apparatus 2 in the third embodiment is designed so as to solve 
soch a disadvantage and is enhanced so that the proportional coefficients a P , a B , a,, a,', etc. shown tn Equation 5 and 
E qu^t»on 6 etc. are adaptively adjusted every moment in accordance with the pattern etc. of the video data so that the 
r«vii drfticutty etita Dj can be approximated by the indicator data with a higher precision than that in the second embod- 
t$ men' and oompf ossed video data having a higher quality can be produced. 

bck*% trie pf ocossing of the video data compression apparatus 2 in the third embodiment will be briefly explained. 
Wncncvor tno encoder 18 of the video data compression apparatus 2 (Fig. 6) ends one picture's worth of com- 
pression and coonc, the host computer 20 learns one picture's worth of the amount of data of the generated com- 
pressed video a*:a and the average value of the quantization values Qj at the time of the compression and coding and 
20 trio global complexity explained below can be calculated. 

Trie global complexity is defined as shown in the following Equation 10-1 to Equation 10-3 as a value obtained by 
mutnalying the amount of data of the compressed video data and the quantization value q in TM5 of the MPEG and 
indicates the complexity of the pattern of the image. 
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X^S.xQ, 00-1) 
Xp = S p xQ p (10-2) 



X B = S B xQ 8 dO-3) 



Note that, in Equation 10-1 to Equation 10-3, S,, S B , and S P indicate the generated amounts of data of the coded 
I picture, B picture, and P picture, Q,, Q B , and Q P indicate the average values of the quantization values Qj when 
producing the I picture, B picture, and P picture, and X,, X b , and X P indicate the global complexity of the I picture, B 
picture, and P picture, respectively. 

The global complexities shown in Equation 10-1 to Equation 10-3 do not always coincide with the real difficulty 
data D i( but substantially coincide with the real difficulty data Dj so far as the average value of the quantization values 
Gj is not extremely large or small. 

Here, when the indicator data of the I picture, P picture, and the B picture, for example the intra AC (other parameter 
is also possible) and ME residual, and the global complexity have a proportional relationship, the proportional coeffi- 
cients e 1 , e p , and e B between these indicator data and global complexity can be calculated by following Equation 11-1 
to Equation 11-3. 



Case of I picture *j= ln J_ AC < 1 1 " 1 > 

Case of P picture ef= Mg resid 0 1 " 2) 



55 Case of B picture ( 11 ' 3 > 
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The real difficulty data Dj of each picture type is calculated as shown in following Equation 12-1 to Equation 12-3 
by using the proportional coefficients e', e p , and e B calculated by Equation 11-1 to Equation 11-3. 

Case of I picture D y= e 7 ' x lntra__ACj (12-1) 



Case of P picture D^-z^x ME_resid } (1 2-2) 

Case of B picture D- =tj x ME_resid } (12-3) 

The host computer 20 calculates and optimizes the proportional coefficients e 1 , e p , and e B whenever the encoder 
18 compresses and codes one picture as shown in Equation 11-1 to Equation 11-3 and finds the values of the real 
15 difficulty data Dj of picture types by Equation 1 2-1 to Equation 1 2-3, whereby the real difficulty data Dj can be always 
optimally approximated by the indicator data irrespective of the pattern of the video data. 

The host computer 20 performs the computation processing shown in Equation 1 with respect to the real difficulty 
data Dj approximated as shown in Equation 11 and Equation 12 to calculate the target amount of data Tj. 

Note that, as in TM5 of the MPEG, when the value of the target amount ot data Tj to be actually calculated is 
20 intentionally changed by a constant ratio with respect to the value determined based on the real difficulty data Dj, the 
target amount of data Tj can be calculated by the following Equation 1 3-1 to Equation 1 3-3: 



25 



30 



Case of I picture (13-1) 
T, *, 

P _picn*rt B _picfurm 



Case of P picture (13-2) 



, 

P _pk*iiTt B_picntrw 

40 

Case of B picture (13-3) 



45 



SO 



KrD, 

r r xRj 



Note that, in all denominators of Equation 1 3-1 to Equation 1 3-3, D, P B indicates the real difficulty data Dj approx- 
imated by the indicator data produced from L pictures' worth of the noncompressed video data buffered in the FIFO 
memory 160 before input to the encoder 1 8, and Rj indicates the average value of the amounts of data which can be 
assigned to L pictures after the j-th picture. 
55 Below, the operation of the video data compression apparatus 2 according to the third embodiment will be ex- 

plained. 

Figure 12 is a view of the compression and coding operation of the video data compression apparatus 2 (Fig. 6) 
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in the third embodiment. 

The encoder control unit 22 rearranges the pictures of the noncompressed video data VIN in the order of coding 
similar to the second embodiment, performs the picture field transformation, etc. and calculates the statistical amount 
such as flatness and intra AC from the j+N-th picture to be compressed and coded to the I picture. 
5 The motion detector 1 4 produces a motion vector for the j+N-th picture to be compressed and coded to the I picture 

or B picture similar to that in the first embodiment and second embodiment and further calculates the ME residual (Fig. 
12A). 

The Fl FO memory 1 60 delays the input video data by exactly the amount of L pictures similar to the first embodiment 
and the second embodiment. 

w The host computer 20 performs the computation processing shown in Equation 12-1 and 12-2 with respect to the 

ME residual produced by the motion detector 14, approximates the real difficulty data Dj. performs the computation 
processing shown in Equation 12-3, and approximates the real difficulty data Dj by the intra AC, etc. (Fig. 12B). 

Further, the host computer 20 substitutes the approximated real difficulty data Dj into Equation 1 or Equations 1 3-1 
to 1 3-3 to calculate the target amount of data Tj and sets the same in the quantization control circuit 1 80 of the encoder 

is 18 (Fig. 12C). 

The DCT circuit 1 66 of the encoder 1 8 performs the DCT processing for the Bh picture of the delayed video data 
similar to that in the first embodiment and the second embodiment. 

The quantization circuit 168 quantizes the data of the frequency domain of the j-th picture input from the DCT 
circuit 166 by the quantization value adjusted based on the target amount of data Tj by the quantization control 
20 circuit 1 80 and, at the same time, calculates the average value of the quantization values Qj used for the compression 
and coding of the j-th picture and outputs the same to the host computer 20. 

The variable length coding circuit 170 performs variable length coding on the quantized data of the j-th picture 
input from the quantization circuit 168 similar to that in the first embodiment and second embodiment to produce the 
compressed video data VOUT of an amount of data near the target amount of data Tj and outputs the same via the 
25 buffer memory 182. 

When the encoder 18 terminates the compression and coding of the j-th picture, the host computer 20 calculates 
the global complexity as shown in Equation 10-1 to Equation 10-3 based on the average value of the quantization 
values Qj with respect to the j-th picture input from the quantization control circuit 180 and the amount of data of the 
compressed and coded j-th picture (Fig. 12D). 
30 Further, the host computer 20 updates the proportional coefficients e', e p , and e B as shown in Equation 11-1 to 

Equation 11-3 by the calculated global complexity (Fig. 12E). The updated proportional coefficients e», e p , and e B are 
reflected in the transformation equations at the time of compression and coding of the next picture (Equation 12-1 to 

Equation 12-3). . . JL . , - 

The content of the processing of the host computer 20 in the third embodiment will be further explained by referring 

35 to Fig. 13. 

Figure 1 3 is a view of the content of the processing of the host computer 20 (Fig. 6) of the video data compression 
apparatus 2 in the third embodiment. 

As shown in Fig. 1 3, at step 300 (S300), the host computer 20 fetches the (j+L)-th indicator data (statistical amount) 
such as ME residual or intra AC from the encoder control unit 22 or the motion detector 14. 
40 At step 302 (S302), the host computer 20 decides to which picture type the G+1)-th picture is to be compressed 

and coded. When the (j+1)-th picture is to be compressed and coded to an I picture, the operation routine proceeds 
to the processing of S304; when it is to be compressed and coded to a P picture, the operation routine proceeds to 
the processing of S306; and when it is to be compressed and coded to a B picture, the operation routine proceeds to 
the processing of S308. 

45 in each ol step 304 (S304), step 306 (S306), and step 308 (S308), the host computer 20 approximates the real 

difficulty data D by Equation 12-1 to Equation 12-3. 

At step 310 (S310), the host computer 20 calculates the target amount of data Tj by Equation 1 or Equation 1 3-1 
to Equation 13-3 by using the approximated real difficulty data Dj. 

At step 312 (S312), the encoder 18 compresses and codes the j-th picture. 
so At step 314 (S314), the host computer 20 calculates the global complexities Xj, X b , and X p [X (I, B, P)] from the 

amount of data of the j-th picture compressed by the encoder 18 and the average value of the quantization values Qj 
set in the quantization circuit 168 by the quantization control circuit 180. 

At step 316 (S316), the host computer 20 decides to which picture type the (j+1)-th picture is to be compressed 
and coded. Where the (j+1)-th picture is to be compressed and coded to an I picture, the operation routine proceeds 
ss to the processing of S318; when it is to be compressed and coded to a P picture, the operation routine proceeds to 
the processing of S320; and when it is to be compressed and coded to a B picture, the operation routine proceeds to 
the processing of S322. 

In each of step 318 (S318). step 320 (S320)m and step 322 (S322), the host computer 20 updates the proportional 
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coefficients e 1 , e p . and e B by Equation 11-1 to Equation 11-3. 

At step 324 (S324). the host computer 20 increments the numerical value j. 

Note that, as mentioned in the second embodiment, there is a case where an offset (5 P ) exists between the real 
difficulty data Dj and the multiplied value of the proportional coefficients e 1 , e p , and e B and the indicator data. In such 
5 a caso. by dividing the values obtained by subtracting the offset values 5', S p , and 5 B from the global complexities X v 
\ by the indicator data, the proportional coefficients e 1 , e p , and e B can be calculated. 
Further, modifications similar to those shown in the second embodiment are also possible for the operation of the 
vkJoo data compression apparatus 2 shown in the third embodiment. 

As mentioned above, according to the operation of the video data compression apparatus 2 in the third embodiment, 
to mo same effect as that by the operation of the video data compression apparatus 2 shown in the second embodiment 
i<, obtained and, in addition, a further correct target amount of data Tj than that in the second embodiment can be 
chelated, and as a result, the quality of the compressed video data can be improved. 

fourth Embodiment 

Below a tsurlh embodiment of the present invention will be explained. 

Up to here the explanation was made of the feed forward rate control (FFRC) system for achieving both an im- 
provement of quahty of the compressed video data and a real time property of the compression and coding processing 
Dy using the indicator data (statistical amount), that is, flatness, intra AC, activity, and ME residual, as the second 
20 embodiment and the third embodiment. 

In the fourth embodiment, an explanation will be made of a video data compression method (improved FFRC 
method) using a video data compression apparatus 2 (Fig. 16 and Fig. 17) with which the encoder for obtaining the 
real difficulty data D, is unnecessary and in addition the quality of the compressed video data of the border (scene 
change) part of the video data (scene) contained in the edited video data is not lowered. 
25 u can be considered to perform the compression and coding by detecting a part where the real difficulty data Dj 

largely changes along with time as a scene change part and changing the picture type sequence. The detection of 
such a scene change is possible also in the FFRC method by monitoring the change of the real difficulty data Dj 
approximated by the indicator data in place of the real difficulty data Dj along with the elapse of time. 

However, to determine the existence of a scene change, it is necessary to monitor the change of the indicator data 
30 within a range of about 1 GOP in time before and after the scene change part. In the video data compression apparatus 
2. the detection of the scene change part becomes possible only after a considerable time has elapsed after the motion 
detector 14 calculates the indicator data. In actuality, there also exists a possibility that the detection of the scene 
change part will become possible only immediately before the compression and coding processing in the encoder 18. 
Accordingly, the host computer 20 must substantially terminate the processing for approximating the real difficulty 
35 data Dj by the indicator data (Equations 5 to 8 shown in the second embodiment and Equation 1 2-1 to Equation 1 2-3 
shown in the third embodiment) before the detection of the scene change in order to ensure sufficient processing time. 

The video data compression apparatus 2 in the fourth embodiment provisionally performs the processing for ap- 
proximation of the real difficulty data Dj by the indicator data or the global complexity in a state where the result of 
detection of the scene change is not determined, corrects only the part requiring a change along with a scene change 
40 in the provisionally calculated real difficulty data Dj after the existence of a scene change and the existence of a change 
of the picture type sequence are determined, and thereby performs the processing for calculating the target amount 
of data Tj. 

Below, the content of the compression and coding processing of the video data compression apparatus 2 in the 
fourth embodiment will be explained by taking as an example a case where the picture type sequence with respect to 
45 n number of pictures is finally determined whenever the calculation of the ME residuals of N [for simplification of the 
explanation, below, for example it is assumed that N = L (L is the number of pictures corresponding to the delay time 
of the FIFO memory 160)] number of pictures is carried out. Note thai, the N number of pictures used for the determi- 
nation of the picture type sequence are the processing units of the processing for determining the picture type sequence 
and do not always have to coincide with the picture type sequence in the encoder 18. Further, the header does not 
so have to be an I picture unlike the usual GOP. Further, below, one set of such N number of pictures will be also referred 
to as a rate control GOP (RCGOP). 

Figure 14 is a view of the compression and coding operation of the video data compression apparatus 2 in the 
fourth embodiment (Fig. 5) in the order of coding. 

The motion detector 14 generates a motion vector for the (j+N)-th picture to be compressed and coded to a P 
55 picture and a B picture similar to that in the first embodiment to the third embodiment and further calculates the ME 
residual. 

The encoder control unit 22 performs the preprocessing of the rearrangement etc. of pictures similar to that in the 
second embodiment and the third embodiment and further calculates the indicator data such as the flatness, intra AC, 
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and activity. 

The FIFO memory 160 delays the input video data by exactly an amount of L pictures similar to that in the first 
embodiment to the third embodiment. 

Whenever the video data compression apparatus 2 (Fig. 5) ends one picture's worth of compression and coding. 
5 the host computer 20, similar to that in the second embodiment and the third embodiment, receives as input the flatness, 
intra AC, and activity calculated by the encoder control unit 22 and the MR residual (statistical amount) calculated by 
the motion detector 14. The host computer 20 stores these indicator data (Fig. 14A). Further, assuming that no scene 
change occurs and no change will occur in the picture sequence, the host computer 20 approximates and predicts the 
real difficulty data Dj in the case assuming that no scene change exists by Equation 12-1 to Equation 12-3 by using 
10 the optimized proportional coefficients e', e p , and e B (Equation 11-1 to Equation 11-3 shown in the third embodiment) 
similar to that in the third embodiment (Fig. 14B). 

Specifically, assuming that the N-th picture from the I picture of the first RCGOP is compressed and coded to an 
I picture, the pictures of the order of whole multiples of M (n x M) are compressed and coded to P pictures, and the 
pictures other than them are compressed and coded to B pictures, the host computer 20 substitutes the indicator data 
is produced by the pictures to be respectively compressed and coded to an I picture, P picture, and B picture and the 
proportional coefficients e 1 , e p , and e B into Equation 1 2-1 to Equation 1 2-3 to approximate and calculate the real difficulty 
data D-. Note, M indicates the interval of the P pictures in the encoder 18 where there is no scene change. 

That is, the host computer 20 counts the number of pictures by using the I picture of the previous RCGOP (first 
CGOP: RCGOP # 1) as a reference, assumes which picture type the encoder 18 compresses and codes each picture 
20 of the second RCGOP (RCGOP#2) to, and approximates and predicts Ihe value of the real difficulty data Dj by the 
indicator data in accordance with the assumed picture type as shown in Equation 12-1 to Equation 12-3. 

Note that, the probability of existence of the scene change part in the RCGOP is considered to be relatively small, 
therefore the host computer 20 will calculate the target amount of data T| with respect to most RCGOP based on the 
predicted real difficulty data Dj (Fig. 14F). 
25 Further, the real difficulty data Dj is only used in the calculation of denominator of Equation 1 (first embodiment) 

or Equation 13-1 to Equation 13-3 (third embodiment). As will be explained later, the host computer 20 performs cor- 
rection at a stage where the existence of a change of the picture type sequence is determined, therefore the value of 
the target amount of data Tj can be always correctly calculated. 

When the calculation of the real difficulty data Dj of each picture of the second RCGOP (RCGOP # 2) is terminated, 
30 the host computer 20 can detect a scene change in the second RCGOP by using the calculated real difficulty data Dj 
or indicator data. For example, the host computer 20 monitors the change of the value of the real difficulty data Dj along 
with the elapse of time and determines that a scene change is generated in a P picture by the method of detecting the 
P picture in which the value of the real difficulty data Dj indicates a value of for example 1 .5 (practically, preferably a 
value within a range of from 1 .4 to 1 .8) times the real difficulty data D M of the P picture immediately before this, and 
35 so on. Where the scene change is detected, the host computer 20 further controls the encoder 18 and changes the 
picture type sequence when compressing and coding the parts before and after the scene change of the edited video 
data so as to change the first P picture of the back scene to an I picture not referring to the last picture of the previous 
scene and change the last I picture of the previous scene to the P picture as shown in Figs. 1 5B and 1 5C. 

By such processing of the host computer 20, the existence of a change of the picture type sequence is seen, and 
40 the picture type to which each picture is -to be compressed and coded is decided (Fig. 14C). 

The host computer 20 corrects the value of the real difficulty data Dj for the second RCGOP based on the stored 
indicator data and the picture type after the change where there is a change in the picture type sequence, calculates 
the correct real difficulty data Dj (Fig. 1 4D), calculates the target amount of data T^ (target bit) of the (N+1 )-th picture 
in accordance with each picture type by using Equation 1 and Equation 1 3-1 to Equation 1 3-3 (Fig. 1 4E), and sets the 
45 same in the quantization control circuit 180 of the encoder 18. 

Specifically, as shown in Fig. 15C, the host computer 20 substitutes the indicator data of the picture changed so 
as to become not the P picture, but the I picture after the compression into Equation 12-2 in place of Equation 12-1 , 
and conversely, substitutes the indicator data of the picture changed so as to become not the I picture, but the P picture 
after the compression into Equation 12-1 in place of Equation 12-2, thereby to correct the value of the real difficulty 
so data Dj. 

The DCT circuit 1 66 of the encoder 1 8 performs DCT processing similar to that in the first embodiment to the third 
embodiment. 

The quantization circuit 168 quantizes the DCT-processed video data by the quantization value Qj adjusted by the 
quantization control circuit 180 based on the target amount of data Tj and calculates the average value of the quanti- 
55 zation values Qj. 

The variable length coding circuit 170 performs variable length coding similar to that in the first embodiment to the 
third embodiment, produces the compressed video data VOUT of an amount of data near the target amount of data 
T:, and outputs the same via the buffer memory 182. 
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When the encoder 18 terminates the compression and coding of the j-th picture, the host computer 20 calculates 
the global complexity as shown in Equation 10-1 to Equation 10-3 based on the average value of the quantization 
values Qj. and the amount of data of the compressed and coded j-th picture. 

Further, the host computer 20 updates and optimizes the proportional coefficients e', e p . and e B as shown in Equa- 
5 tion 11 -1 to Equation 1 1 -3 by the calculated global complexity. In the same way as in the third embodiment, the updated 
proportional coefficients e 1 , e p , and e B are reflected in the transformation equations at the time of compression and 
coding of the next picture (Equation 12-1 to Equation 12-3). 

The content of the processing of the host computer 20 in the fourth embodiment will be further explained referring 
to Fig. 16. 

io Figure 16 is a flowchart of the content of the processing of the host computer 20 in the video data compression 

apparatus 2 in the fourth embodiment. 

As shown in Fig. 16, the processing of the host computer 20 in the fourth embodiment is divided into a first stage 
(S400) and a second stage (S420). At the first stage, processing for predicting the real difficulty data Dj is carried out 
by assuming that there is no scene change and there is no change in the picture type sequence, and at the second 

is stage, processing for correcting the value of the real difficulty data Dj is carried out where a scene change occurs and 
the picture type sequence is changed. 

The first stage (S400; S402 to S412) is the processing for predicting the real difficulty data Dj where there is no 
scene change. At step 402 (S402) of the first stage, the host computer 20 fetches the indicator data (statistical amount) 
such as the ME residual or intra AC of the (j+L)-th picture from the encoder control unit 22 or the motion detector 14 

20 and stores the same. 

At step 404 (S404), the host computer 20 decides whether or not the [j+L (j+N)]-th picture is to be compressed 
and coded to a B picture. Where the (j+L)-th picture is to be compressed and coded to a B picture, the operation routine 
proceeds to the processing of S406, while where it is not to be compressed and coded to a B picture, the operation 
routine proceeds to the processing of S408. 
25 At step 406 (S406), the host computer 20 predicts that the (j+L)-th picture is to be compressed and coded to a B 

picture and approximates and calculates the real difficulty data Dj by Equation 12-3. 

At step 408 (S408), the host computer 20 decides whether or not the number (interval) from the picture to be 
compressed and coded to the 1 picture in the previous RCGOP to the (j+L)-th picture of the current RCGOP is N. Where 
the interval is N, the operation routine proceeds to the processing of S412, while where it is not N. the operation routine 
30 proceeds to the processing of S41 0. 

At step 410 (S410), the host computer 20 predicts that the G+L)-th picture is to be compressed and coded to a P 
picture and approximates and calculates the real difficulty data Dj by Equation 12-2. 

At step 412 (S412). the host computer 20 predicts that the (j+L)-th picture is to be compressed and coded to an I 
picture and approximates and calculates the real difficulty data Dj by Equation 12-1. 
35 The second stage (S420; S422 to S434) is the processing for correcting the real difficulty data Dj predicted at the 

first stage. At step 422 (S422) of the second stage, the host computer 20 decides whether or not a new RCGOP starts. 
Where it does not start, the operation routine proceeds to the processing of S430, while where it starts, the operation 
routine proceed to the processing of S424. 

At step 424 (S424), the host computer 20 decides whether or not the picture type sequence is changed so that 
40 the position of the I picture changes. Where the picture type sequence is changed so that the position of the I picture 
changes, the operation routine proceeds to the processing of S426, while where it is not changed, the operation routine 
proceeds to the processing of S430. 

At step 426 (S426). the host computer 20 approximates and calculates the real difficulty data Dj by Equation 12-1 
for the picture newly compressed and coded to the I picture. 
45 At step 428 (S428), the host computer 20 approximates and calculates the real difficulty data Dj by Equation 1 2-2 

for the picture newly compressed and coded to the P picture. 

At step 430 (S430), the host computer 20 calculates the target amount of data Tj with respect to the j-th picture 
by Equation 1 or Equation 1 3-1 to Equation 1 3-3 and sets the same in the quantization control circuit 180 of the encoder 
18 (Fig. 5). 

so At step 432 (S432), the encoder 18 compresses and codes the j-th picture based on the target amount of data Tj 

set in the quantization control circuit 180. 

At step 434 (S434), the host computer 20 increments the numerical value j. 

Note that, in the fourth embodiment, the host computer 20 of the video data compression apparatus 2 performs 
the processing for correcting only the real difficulty data Dj of pictures changed after compression where there is a 
55 scene change, but when there is a room in the processing time, it can be modified so that the real difficulty data Dj of 
all pictures are calculated after the picture type sequence is decided. 

Further, modifications similar to those in the first embodiment to the third embodiment are also possible for the 
operation of the video data compression apparatus 2 shown in the fourth embodiment. 
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Further, the contents of the processing of the video data compression apparatuses 1 and 2 respectively explained 
in the first embodiment to the third embodiment can be combined so far as they do not contradict each other. 

As mentioned above, according to the operation of the video data compression apparatus 2 in the fourth embod- 
iment, the same effect as that by the operation of the video data compression apparatus 2 shown in the second em- 
5 bodirnent and the third embodiment is obtained: a further correct target amount of data Tj than that in those embodi- 
ments can be calculated, and in addition the quality of the compressed video data of the scene change part is not 
lowered. 

As explained above, by the video data compression apparatus according to the present invention and the method 
of the same, audio and/or video data can be compressed and coded to a predetermined amount of data or less by a 
10 method other than the two pass encoding. 

Further, by the video data compression apparatus according to the present invention and the method of the same, 
the video data can be compressed and coded in almost real time and in addition a high quality image can be obtained 
after expansion and decoding. 

Further, by the video data compression apparatus according to the present invention and the method of the same, 
is the compression rate is adjusted by estimating the amount of data after the compression and coding and the compres- 
sion and coding processing can be carried out by a method other than two pass encoding. 



Claims 

20 

1. A video data compression apparatus comprising: 

an indicator data calculating means for calculating indicator data indicating a complexity of video data for every 
picture from noncompressed video data; 
25 a target value calculating means for calculating a target value of an amount of data after compression of said 

video data for every picture based on said calculated indicator data; and 

a compressing means for compressing said noncompressed video data so that the amount of data after com- 
pression becomes said calculated target value. 

30 2. A video data compression apparatus according to claim 1 , wherein: 

said compressing means compresses said video data to a pictu re type sequence containing a plurality of types 
of pictures (I picture, P picture, and B picture or a combination of them) in a predetermined order; 
said indicator data calculating means calculates an ME residual as said indicator data of the pictures to be 
35 compressed to the P picture and B picture and calculates a flatness and an intra AC data or one of the same 

as said indicator data of a picture to be compressed to an I picture; and 

said target value calculating means calculates difficulty data corresponding to the amount of data after com- 
pression based on said calculated indicator data and further calculates said target value based on the calcu- 
lated difficulty data. 

40 

3. A video data compression apparatus according to claim 1 , wherein: 

said indicator data calculating means calculates an activity as said indicator data of the I picture of said video 

data. 

45 4. A video data compression apparatus according to claim 1 , further comprising 

a delaying means for delaying said video data for a predetermined time and then outputting the same; 
said target value calculating means calculates said target value with respect to a picture output by said delaying 
means based on said indicator data calculated during a period where said delaying means delays said video 
so data; and 

said compressing means compresses pictures output by said delaying means so that the amount of data after 
compression becomes said calculated target value. 

S. A data compression method comprising the steps of: 

55 

calculating indicator data indicating a complexity of video data for every picture from noncompressed video 
data; 

calculating a target value of an amount of data after compression of said video data for every picture based 
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on said calculated indicator data; and 

compressing said video data by a predetermined compression method so that the amount of data after com- 
pression becomes said calculated target value. 

5 6. A video data compression method according to claim 5, wherein: 

said compressing step compresses said video data to a picture type sequence containing a plurality of types 
of pictures (I picture, P picture, and B picture or a combination of the same) in a predetermined order; 
said indicator data calculating step calculates an ME residual as said indicator data of pictures to be corn- 
to pressed to a P picture and B picture and calculates a flatness and intra AC data or one of the same as said 
indicator data of a picture to be compressed to an I picture; 

said data amount target value calculating step further has a step for calculating difficulty data corresponding 
to the amount of data after compression based on said calculated indicator data; and 
said target value is calculated based on the calculated difficulty data. 

1S 

7. A video data compression method according to claim 5, wherein: 

said indicator data calculating step calculates an activity as said indicator data of the I picture of said video 

data. 

20 8. A video data compression method according to claim 5, wherein: 

it further comprises a step of delaying said video data by a predetermined time and then outputting the same; 
said data amount target value calculating step calculates said target value with respect to a picture delayed 
and output based on said indicator data calculated during a period where said video data is delayed; and 
25 said compression step compresses the delayed and output picture so that the amount of data after compres- 

sion becomes said calculated target value. 

9. A video data compression apparatus comprising: 

30 an indicator data calculating means for calculating indicator data indicating a complexity of videodata for every 

picture; 

a difficulty data calculating means for performing a predetermined computation processing for multiplying a 
coefficient with said calculated indicator data to calculate difficulty data corresponding to the amount of data 
after compression of said video data; 
35 a target value calculating means for calculating a target value of the amount of data after compression of said 

video data for every picture based on said calculated difficulty data; 

a compressing means for compressing each of the pictures of said video data by a predetermined compression 
method so that the amount of data after compression becomes said calculated target value so as to generate 
compressed video data; and 

40 a coefficient updating means for updating said coefficient based on the amount of data of the generated com- 

pressed video data. 

10. A video data compression apparatus according to claim 9, wherein: 

45 said compressing means compresses said noncompressed video data to a picture type sequence containing 

a plurality of types of pictures (I picture, P picture, and B picture or combination of the same) in a predetermined 
order; and 

said indicator data calculating means calculates an ME residual as said indicator data of pictures to be com- 
pressed to a P picture and B picture and calculates a flatness, intra AC data, and activity or a combination of 
so the same as said indicator data of a picture to be compressed to an I picture. 

11. A video data compression apparatus according to claim 10, wherein: 

said compressing means has 

55 a quantizing means for quantizing said video data by a quantization value set from an external unit so as to 

generate said compressed video data and 

a quantization value adjusting and setting means for successively adjusting said quantization value based on 
said calculated target value and setting the same in said quantizing means; and 
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said coefficient updating means updates said coefficient based on an average value of said quantization values 
set in said quantizing means of said compressing means, an amount of data of said generated compressed 
video data, and said calculated indicator data. 

5 12. A video data compression apparatus according to claim 11 , wherein 

said coefficient updating means has: 

a global complexity calculating means for calculating a global complexity based on the average value of said 
quantization values set in said quantizing means of said compressing means and the amount of data of said 
w generated compressed video data and 

a coefficient calculating means for calculating said coefficient based on said calculated global complexity and 
said indicator data. 

13. A video data compression apparatus according to claim 12, wherein: 
t5 said coefficient calculating means divides the global complexity of a picture which becomes an I picture after 

compression by said generated flatness, intra AC, or activity to calculate said coefficient for an I picture and divides 
the global complexity of a picture which become a P picture or a B picture after compression by said generated 
ME residual to calculate said coefficient for a P picture and said coefficient for a B picture. 

20 14. A video data compression apparatus according to claim 13, wherein: 

said coefficient calculating means adds or subtracts a predetermined offset value with respect to said global 
complexity and divides the result by said generated flatness, intra AC, or activity to calculate said coefficient for 
an I picture and divides the global complexity of a picture which becomes a P picture or a B picture after compression 
by said generated ME residual to calculate said coefficient for a P picture and said coefficient for a B picture. 

25 

15. A video data compression method comprising the steps of: 

calculating indicator data indicating a complexity of video data for every picture; 

performing predetermined computation processing for multiplying a coefficient with said calculated indicator 
30 data to calculate difficulty data corresponding to the amount of data alter compression; 

calculating a target value of the amount of data after compression of said noncompressed video data for every 
picture based on said calculated difficulty data; 

compressing each of the pictures of said video data by the compression method so that the amount of data 
after compression becomes said calculated target value so as to generate compressed video data; and 
35 updating said coefficient based on the amount of data of the generated compressed video data. 

16. A video data compression method according to claim 15, wherein: 

said compressing step compresses said video data to a picture type sequence containing a plurality of types 
40 of pictures (I picture, P picture, and B picture or combination of the same) in a predetermined order; and 

said indicator data calculating step calculates an ME residual as said indicator data of pictures to be com- 
pressed to a P picture and B picture and calculates a flatness, intra AC data, and activity or a combination of 
the same as said indicator data of a picture to be compressed to an I picture. 

45 17. A video data compression method according to claim 16, wherein: 

said compressing step further contains 

a step of quantizing the video data subjected to said predetermined compression processing by a quantization 
value set from an external unit so as to generate said compressed video data and 
so a step of successively adjusting and setting said quantizatbn value based on said calculated target value; and 

said updating step updates said coefficient based on an average value of said adjusted and set quantization 
values, the amount of data of said generated compressed video data, and said calculated indicator data. 

18. A video data compression method according to claim 17, wherein: 

ss 

said updating step 

calculates a global complexity based on the average value of said adjusted and set quantization values and 
the amount of data of said generated compressed video data and 
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calculates said coefficient based on said calculated global complexity and said indicator data. 

19. A video data compression method according to claim 1 8, wherein: 

said updating step divides the global complexity of a picture which becomes an I picture after compression 
5 by said generated flatness, intra AC, or activity to calculate a coefficient for the I picture and divides the global 

complexity of a picture which becomes a P picture or a B picture after compression by said generated ME residual 
to calculate said coefficient for a P picture and said coefficient for a B picture. 

A video data compression method according to claim 1 9, wherein: 

said updating step adds or subtracts a predetermined offset value with respect to said global complexity and 
divides the result by said generated flatness, intra AC, or activity to calculate said coefficient for an I picture and 
divides the global complexity of a picture which becomes a P picture or a B picture after compression by said 
generated ME residual to calculate said coefficient for a P picture and said coefficient for a B picture. 

is 21. A video data compression apparatus for compressing a continuous plurality of video data to compressed video 
data of a picture type sequence containing a plurality of types of pictures (I picture, P picture, and B picture) in a 
predetermined order, comprising: 

a rearranging means for rearranging pictures of said noncompressed video data to an order adapted to the 
20 compression method so that each head picture of said video data becomes an I picture or a P picture; 

an indicator data calculating means for calculating indicator data indicating a complexity of said rearranged 
noncompressed video data for every picture; 

a border detecting means for detecting a scene change of a continuous plurality of said noncompressed video 
data; 

25 a changing means for changing said picture type sequence so that pictures of any of said noncompressed 

video data are compressed without reference to the pictures of said other noncompressed video data for every 
border of a detected scene change; 

a target value calculating means for calculating a target value of the amount of data after compression of said 
video data based on said calculated indicator data and said picture type sequence after change; and 
30 a compressing means for compressing said video data to compressed video data of said picture type sequence 

after change so that the amount of data after compression becomes substantially said calculated target value. 

22. A video data compression apparatus according to claim 21 , wherein: 

as said target value calculating means has 

an approximating means for performing predetermined computation processing for multiplying a coefficient 
with said calculated indicator data to calculate difficulty data corresponding to the amount of data after com- 
pression and 

a calculating means for calculating a target of the amount of data after compression of said noncompressed 
40 video data for every picture based on said calculated difficulty data; 

said compressing means has 

a quantizing means for quantizing said video data by a quantization value set from an external unit so as to 
generate said compressed video data and 

a quantization value adjusting and setting means for successively adjusting said quantization values based 
45 on said calculated target value and setting the same in said quantizing means; and 

provision is further made of a coefficient updating means for updating said coefficient based on the average 
value of said quantization values set in said quantizing means of said compressing means, the amount of data 
of said generated compressed video data, and said calculated indicator data. 

so 23. A video data compression apparatus according to claim 21 , wherein 

said target value calculating means has: 

a predictive target amount calculating means for calculating said target value in accordance with the type of 
picture after compression by predicting that pictures contained in the predetermined compression unit are 
55 compressed as an order of said picture type sequence in advance before the change of said picture type 

sequence and 

a target amount correcting means for correcting said target value of a picture of said video data of a type of 
picture which after compression is changed in accordance with the type of the picture after the change in only 



20 

BNSDCCID: <EP 0825782A2_I_> 



EP 0 825 782 A2 

a case where a change of said picture type sequence actually exists. 

24. A video data compression apparatus according to claim 23, wherein: 

5 said indicator data calculating means calculates a flatness, intra AC, and activity as indicator data of a picture 

which becomes an I picture after compression and an ME residual as indicator data of a picture which becomes 
a P picture or a B picture after compression; 

said changing means changes said picture type sequence so that the picture of the head of said video data 
is compressed to an I picture when the picture of the head of said video data would be compressed to a P 
10 picture; and 

said target amount correcting means corrects said target value of a picture of a type of picture after compression 
which changes from a P picture to an I picture, which is calculated in advance, to said target amount of a 
picture in a case where it becomes an I picture after compression and corrects said target value of a picture 
of a type of the picture after compression which changes from an I picture to a P picture, which is calculated 
is in advance, to said target amount of a picture in a case where it becomes a P picture after compression. 

25. A video data compression apparatus according to claim 22, wherein 

said coefficient updating means has 
20 a global complexity calculating means for calculating a global complexity based on an average value of said 

quantization values set in said quantizing means of said compressing means and the amount of data of said 
generated compressed video data and 

a coefficient calculating means for calculating a coefficient based on said calculated global complexity and 
said indicator data. 

25 

26. A video data compression apparatus according to claim 25, wherein: 

said coefficient calculating means divides a global complexity of a picture which becomes an I picture after 
compression by said generated flatness, intra AC, or activity to calculate a coefficient for an I picture and divides 
a global complexity of a picture which becomes a P picture or a B picture after compression by said generated ME 
30 residual to calculate a coefficient for a P picture or a coefficient for a B picture. 

27. A video data compression method for compressing a continuous plurality of video data to compressed video data 
of a picture type sequence containing a plurality of types of pictures (I picture, P picture, and B picture) in a pre- 
determined order, comprising the steps of: 

rearranging pictures of said noncompressed video data to an order adapted to the compression method so 
that each head picture of said video data becomes an I picture or a P picture; 

calculating indicator data indicating a complexity of said rearranged noncompressed video data for every pic- 
ture; 

detecting a scene change of the continuous plurality of said noncompressed video data; 
changing said picture type sequence so that a picture of any of said noncompressed video data is compressed 
without reference to a pictures of other noncompressed video data for every border of a detected scene change; 
calculating a target value of the amount of data after compression of said video data for every predetermined 
compression unit based on said calculated indicator data and said picture type sequence after change; and 
compressing said video data to the compressed video data of said picture type sequence after change so that 
the amount of data after compression becomes substantially said calculated target value. 

28. A video data compression method according to claim 27, wherein: 

said target value calculating step performs predetermined computation processing for multiplying a coefficient 
with said calculated indicator data so as to calculate difficulty data corresponding to the amount of data after 
compression and 

calculates a target of the amount of data after compression of said noncompressed video data for every picture 
based on said calculated difficulty data; 

said compressing step quantizes video data subjected to said predetermined compression processing by a 
quantization value set from an external unit so as to generates said compressed video data and 
successively adjusts said quantization value based on said calculated target value and sets the same; and 
said predetermined coefficient is updated based on the average value of said set quantization values, the 
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amount of data of said generated compressed video data, and said calculated indicator data. 

29. A video data compression method according to claim 27, wherein 

s said target value calculating step calculates said target value in accordance with the type of picture after 

compression by predicting that pictures contained in said predetermined compression unit are compressed 
as an order of said picture type sequence in advance before the change of said picture type sequence and 
corrects said target value of the picture of said noncompressed video data of a type of picture which after 
compression is changed in accordance with the type of the picture after the change in only a case where a 

10 change of said picture type sequence actually exists. 

30. A video data compression method according to claim 29, wherein: 

said indicator data calculating step calculates a flatness, intra AC, and activity as indicator data of a picture 
is which becomes an I picture after compression and an ME residual as indicator data of a picture which becomes 

a P picture or a B picture after compression; 

changes said picture type sequence so that the picture ol the head of said video data is compressed to an I 
picture where the picture of the head of said video data would be compressed to a P picture; and 
corrects said target value of a picture of a type of picture after compression which is changed from a P picture 
20 to an I picture, which is calculated in advance, to said target amount of a picture in the case where it becomes 

an I picture after compression and corrects said target value of a picture of a type of picture after compression 
which is changed from an I picture to a P picture, which is calculated in advance, to said target amount of a 
picture in the case where it becomes a P picture after compression. 

25 31. A video data compression method according to claim 28, wherein 

said coefficient updating step calculates a global complexity based on the average value of said quantization 
values to be set and the amount of data of said generated compressed video data and 
calculates said coefficient based on said calculated global complexity and said indicator data. 

30 

32. A video data compression method according to claim 31 , wherein: 

said coefficient calculating step divides a global complexity of a picture which becomes an I picture after 
compression by said generated flatness, intra AC, or activity to calculate said coefficient for an I picture and divides 
a global complexity of a picture which becomes a P picture or a B picture after compression by said generated ME 
35 residual to calculate said coefficient for a P picture or said coefficient for a B picture. 



40 



45 



SO 



55 



22 

BNSDOCID: <EP 0825782A2_1_> 



EP 0 825 782 A2 



U- 




—1 




RNSOOCID: <EP 08257B2A2 l_> 



23 



EP 0 825 782 A2 




24 

BNSDOCIO. <EP 0825782A2_I_> 



EP 0 825 782 A2 




BNSDOCID: <EP_08257B2A2_I_> 



25 



EP 0 825 782 A2 



/ ~7 
/ 7 



CP 



7 



CN4 
-t- 



~7 



7 



%Z1 



CO 



I 

i c 
i 

i 



BNSDOCIO. <EP 0825782A2_I_> 



26 



EP 0 825 782 A2 






o 




CM 

' \ 










•\SSr- 






• i - 



RNejYTT.irv <FP 0RP57B2A2 I > 



27 



EP 0 825 782 A2 



LL, 




•© 

S 
5 



O 



o- 



BNSDOCID: <EP 0825782A2_I_> 



28 



EP 0 825 782 A2 



FIG. 7 



difficulty* 10 c 



. cheer 

c mobile 

♦ rosort 

* tennis 
o diva 



1.80- 

1.70- 
1.60- 
1.50- 
1. 40 - 
1.30- 
1.20- 
1.10- 
1.00- 
0. 90- 
0. 80- 

0. 70" 
0. 60- 
0. 50- 
0. 40- 
0. 30- 
0. 20- 



m x 

£■ fx 



•■■ ft- 



2. 00 



4. 00 



6. 00 



De_resjcix 10* 
.00 10.00 ' 12.00 14.00 . 



RNSDOOID: <EP 0825782A2 I > 



29 



EP 0 825 782 A2 



FIG. 8 



difficulty xi0. b 



u cheer 

□ mobile 

♦ resort' 

* tennis, 
o diva 
































— ^— 














o 

o. 
















o 

— ft . 
















o 














» 
















o 














O 
















■ o 
o 




























o 

o 
















A° 














.. .. ^ 














— 
































< 


















)0 4. 


00 6. 


00 . B. 


1 '. *• 

00 10.00 12/00 14 


•ae_resid 
00 ' 



1. 40. 
1.30- 
1, 20 - 
1. 10- 
1. 00 
0. 00- 

a so. 

0. 70- 

o. no • 

0. 50- 

0..40- 
0. 30- 

0.20- 
0. 10 - 



A 

SDOCIO <EP 0825782A2_L> 



30 



EP 0 825 782 A2 



FIG. 9 



® v (D 

y<i min[|A-D|. IB-Cl) 



BNSOOCID; <EP 08257B2A2J_> 



31 



EP 0 825 782 A2 



FIG. 10 



difficulty* 10 s 

1.60 



V cheer 
•o mobile- 

o resort 
•"■tenuis 

♦ diva 











































































— o 




























































— 






o 


















Is*-. — 






































































> — 
















3 — 


























■ "It . — 


* 




3- 












x * 




5- 












o 


















0 - 
5- 
























l r— \ - 


X 


0 













80.00* • 100.00 120.-00 140.00 160.00 -180.00 200.00 



BNSDOCIO <EP 082S782A2 1 > 



32 



EP 0 825 782 A2 



FIG. 11 



difficulty*? 10* 



« cheer 

o mobile 
♦ resort 
m. tennis 
o diva 



1.55- 
1.50- 
1.45- 
1. 40- 



1.35- 
1.30- 
1.25- 
1. 20. 
1. 15, 
1. in- 
1.05- 
K 00 

0: 95- 

0. 90- 
0. 85' 
0. 80- 
0. 75- 
0. 70- 
0. 65- 
0. 60- 



1 




_ 1 _ 








: : 










<Jf ■ 






























































V. 










































































q 


k — 














































































-o — : — 






























* 














X 




























— 




I 










































)— 










1 





I oo- 



6. 00 



8. 00 



10.00 ' 12.00. 



U-00 



15. 



intra ACxlO* 
00 ' 18. 00 



BNSDOCID: <EP 08257B2A2 .!_> 



33 



EP 0 825 782 A2 




BNSOOCID: <EP_0825782A2_I_> 



34 



EP 0 825 782 A2 



FiG. 13 



( START ^ 



S3Q0 



•4- ^ 
FETCH INTRA-AC OF ME 
RESIDUAL OF PICTURE' j+N 




■PICTURE TYPE ? 
B-PICTUR 



OIFFICULTY'GONVERSION . 
BY EQUATION (12-3) 



DIFFICULTY CONVERSION 
. BY' EQUATION (1.2-2) 



DIFFICULTY CONVERSION-. 
BY EQUATION -(l2--i > . 



J 



CALCULATE TARGET BIT 
OF PICTURE:]: 
BY EQUATION .(1) 



ENCODING - OF' iPICTURE j 



S310 



^S312 



SEEK GLOBAL .COMPLEXITY' X (I, P, B) 

FROM.- GENERATED BIT AMOUNT 
AND AVERAGE QUANTIZATION .VALUE 



S314 



S316 




S322 PPICTURE 



UPDATE PROPORTIONAL 
CONSTANT 
BY EQUATION (11-3) 



S320 



•PICTURE 



UPDATE PROPORTIONAL 
CONSTANT.- ' 
BY EQUATION (11.-2) • ' 



S318 



UPDATE PROPORTIONAL 
. CONSTANT - 
BY EQUATION 



J"='j + .1 



< i. 

Q RETURN^ ) 



S324. 



RNJSDnClD: <EP 0825782A2 ( > 



35 



EP 0 825 782 A2 




BNSDOCIO. <EP_0825782A2_I_> 



36 



EP 0 825 782 A2 



FIG. 15 



(A) 



(B) 



ELASPE OF REAL TIME (DISPLAY ORDER) 



GOP 



FRONT SCENE 



i SCENE CHANGE 

BACK SCENE 



P B B 



8BPBBPBBPBBPBB 



K 



IN 



NJ 



SJ 



1 



N 



N 



i * 



* i 



P B B 



*-.!••.. *-J/*.. • 
+ f'^'x t 

BBPBBPBBPBBPBBI 




SI 

BACK SCENE 



SCENE CHANGE 



ELASPE OF TIME WHEN ,^^1?^.^^^..^!^^ IMAGE 



(C) 



P B B 



SI 



SI 



P|BBPBBPBB 
IN 



nJ 



si 



s 



SJ 



s] 



FRONT SCENE 



GOP 



B B. P B B 

fSN^ISts 

^Nj 

♦ BACK SCENE 



SCENE CHANGE 



CHANGE I PICTURE 
TO P PICTURE 



CHANGE P PICTURE 
TO I PICTURE 



37 



EP 0 825 782 A2 



FIG. 16 



( START 



400 



S402 



FETCH ME RESIDUAL AND 
INTRA-AC OF PICTURE j+N 




PICTURE TYPE ? 
B : PICTURE 



i 
i 

\ 



FIRST 
STEP 



DIFFICULTY CONVERSION 
BY EQUATION (12-3) 



S422 



DIFFICULTY CONVERSION 
BY EQUATION (12-2) 



DIFFICULTY CONVERSION 
BY EQUATION (12-1) 




S410 



T 

S412 



CALCULATE TARGET BIT 
OF PICTURE j 
BY (1) AND (13) 




ENCODING OF PICTURE j 



j = j + 1 



$432 



S434 



f RETURN 



CALCULATE DIFRCULTY 
BY EQUATION (12-1) 
FOR NEW I PICTURE 



CALCULATE DIFFICULTY 
BY EQUATION (12-2) 
FOR NEW P PICTURE 



S426 



S428 



SECOND 
STEP 



J 



BNSDOCID: <EP 082S782A2_I_> 



38 



